1. Field of the Invention
The invention relates to license plate reading apparatus and method for recognizing a license plate by an image of a vehicle photographed on a road. More particularly, the invention relates to license plate reading apparatus and method for automatically detecting a running vehicle from information of an image photographed by one television camera and reading characters on the license plate at a high precision.
2. Description of the Related Arts
In recent years, a system for recognizing a license plate of a vehicle running a road has socially played an important role. Therefore, there has been developed a system in which a TV camera is put on a road and a license plate of a vehicle is recognized from a photographed image. A license plate reading apparatus has to satisfy two requirements such that, first, all running vehicles can be detected without missing a detection, and second, the license plate of the detected vehicle is correctly read. As a conventional technique for this purpose, for example, there is an apparatus disclosed in JP-B-2893814. In such an apparatus, a head of a running vehicle is detected by extracting an edge of an image. A license plate is read by a method of binarizing the image by a background difference.
However, the method of detecting the vehicle head of the running vehicle by extracting the edge of the image has a drawback such that the vehicle head is easily erroneously discriminated as a shadow of the vehicle, a reflection of headlights, or the like. Also a method of reading a license plate by binarizing an image by a background difference has a problem such that it is difficult to avoid an adverse influence on conditions of external illuminance and a reading precision is low. In order to read the license plate of the vehicle at a high precision, it is necessary to raise a resolution of characters. However, to keep the high resolution, it is necessary to use a plurality of cameras or use a camera of a high resolution, so that the costs become very high. Therefore, in case of using an ordinary TV camera, a passing position such as a gate or the like of the vehicle is extremely limited and the system is limited to an application in which the position of the license plate has been fixed on the image. Therefore, it is difficult to read the license plate of the vehicle which is running a road at a high precision by using the ordinary TV camera.
According to the invention, there are provided license plate reading apparatus and method for certainly detecting a running vehicle from an image of a relatively low resolution photographed by one camera and reading a license plate from the detected image at a high precision.
A license plate reading apparatus of the invention comprises: an image pickup unit (TV camera) which is set above a road and photographs a running vehicle at a predetermined frame period; an image storing unit for successively storing two images of a present frame image and a previous frame image which were photographed by the image pickup unit; a vehicle detecting unit for detecting a motion of a license plate in a head portion of the running vehicle on the basis of a difference between the present frame image and the previous frame image which were stored in the image storing unit; a dictionary in which character images of all characters used for the license plate have been registered; and a character recognizing unit for recognizing all of the characters of the license plate from an image in a license plate area detected by the vehicle detecting unit by correlating with the registered character images in the dictionary. Therefore, even for a video image photographed by one camera by a sight of about two times as wide as a width of an ordinary vehicle, the license plate can be recognized at a high precision.
The vehicle detecting unit comprises: a block dividing unit for dividing each of the present frame image and the previous frame image into blocks of a predetermined pixel size; a vector calculating unit for obtaining a pixel position, as a block moving position, where a sum of differences between each pixel in the block of the previous frame and all pixels in the blocks of the present frame is equal to a minimum value bg, thereby calculating a motion vector of the block; and a vehicle head detecting unit for scanning all of the blocks, searching a group of blocks in which sizes and directions of the motion vectors are almost the same in an area of a rectangular size corresponding to the license plate, detecting the vehicle on the basis of the presence or absence of the group of blocks which satisfy such conditions, and at the same time, detecting a license plate area of the vehicle head. Since the vehicle detection is performed by paying attention only to the motion of the license plate in the image of the running vehicle as mentioned above, the vehicle can be accurately detected without being influenced by an image other than the license plate.
Further, in order to improve the detecting precision of the vehicle, the vehicle detecting unit can also have: a block dividing unit for dividing each of the present frame image and the previous frame image into blocks of a predetermined pixel size; a luminance level threshold value setting unit for always measuring a luminance of a predetermined area on the road in the frame image, discriminating statistical external illuminance state by obtaining an average value of a predetermined time, and obtaining and setting a luminance level threshold value (bg_th) which monotonously increases in accordance with the illuminance state by a predetermined simple increasing curve; a vector calculating unit for obtaining a pixel position, as a block moving position, where a sum of a difference between each pixel in the block of the previous frame and all pixels in the block of the present frame is equal to the minimum value (bg), thereby calculating a motion vector of the block; a reliability amount calculating unit for calculating a motion vector reliability amount (bgm) by subtracting a sum (bg0) of a difference between each pixel in the block of the present frame and all pixels in the block of the previous frame from the minimum difference value (bg) calculated by the vector calculating unit; and a vehicle head detecting unit for scanning all blocks, searching a group of blocks in which sizes and directions of the motion vectors are almost the same and the reliability amount (bgm) is equal to or larger than the luminance level threshold value (bg_th) in an area of a rectangular size corresponding to a license plate, detecting a vehicle on the basis of the presence or absence of the group of blocks which satisfy those conditions, and at the same time, detecting a license plate area at the head of the vehicle. As mentioned above, by always measuring the luminance level of a part of the image, setting the difference of a correlation result between the image and the position where the motion is equal to zero to the reliability amount of the vector, and discriminating the motion from a shadow and noises of the vehicle, the vehicle and its license plate area can be detected accurately from the image.
The character recognizing unit comprises: a low resolution recognizing unit for recognizing a character at a position of the character having the largest size on the license plate with respect to the frame image of a low resolution obtained by thinning out the original image as a target; and a high resolution recognizing unit for returning the resolution of the frame image to the original resolution, recognizing the character at the character position recognized by the low resolution recognizing unit, and recognizing a character at a position of another small character on the license plate in which a relative position has previously been known on the basis of a character recognition result. As mentioned above, by performing the recognition of two stages such that the character recognition of the license plate is performed with respect to the image of a low resolution, as a target, obtained by thinning out the pixels from the image in which the vehicle head has been detected and with respect to candidate positions thus obtained, the resolution is returned to the original resolution and the character recognition is performed, the reading at a high speed and a high precision is realized.
In the dictionary which is used for the character recognition, each of a sample character image obtained by photographing each character which is used for the license plate and a sample character image obtained by thinning out the above sample character image to the image as that of the low resolution recognizing unit is reduced to a size corresponding to the size of the character image at the time of reading the license plate, thereafter, converted into a Laplacian character image by applying a Laplacian filter, and registered into the dictionary. Each of the low resolution recognizing unit and high resolution recognizing unit applies the Laplacian filter to the inputted character image and converts and, thereafter, performs a normalization correlation arithmetic operation between the converted image and the Laplacian character image of the corresponding resolution which has been registered in the dictionary, thereby recognizing the characters of the license plate. The image having the highest correlation degree can be set to the collation result by the normalization correlation calculation between the dictionary character image converted by applying the Laplacian filter and the input character image, and the character recognition can be performed at a high precision.
The high resolution recognizing unit calculates an average value of the luminance of a background portion of the license plate, calculates an accumulation average value for a predetermined time, predicts an external illuminance, and when the accumulation average value is equal to or less than a predetermined threshold value, performs a contrast correction for improving the luminance of the character image to a predetermined level, and performs the character recognition of the license plate. Therefore, even if the ambient brightness decreases, the contrast of the license plate image is automatically improved, thereby improving the character recognizing precision in nighttime or the like. As for the dictionary which is used for the character recognition, further, as dictionary character images of the characters of small sizes used for the license plate, a plurality of kinds, for example, nine kinds of shift character images are formed by shifting the photographed sample character image in the vertical, lateral, and oblique directions and reduced to a size corresponding to the size of the character image at the time of reading the license plate, and thereafter, the images are converted into the Laplacian character images by applying the Laplacian filter and registered into the dictionary. By registering the images obtained by shifting the sample character in the lateral, oblique, and vertical directions as mentioned above into the dictionary, a fine sampling error at the low resolution caused by photographing the image by the ordinary TV camera is absorbed and the characters are recognized at a high precision.
Further, when the sample character is registered into the dictionary, the dictionary image is formed by selecting the Laplacian filter having a structure in which each size and/or the vertical/lateral sizes have been determined in correspondence to the kind (Hiragana, Kanji, or small number) of character of a small size used in the license plate. The high resolution recognizing unit selects the Laplacian filter of each character font used in the dictionary formation and recognizes the characters of the input character image. Therefore, the recognition at the low resolution can be performed at a high precision in accordance with the character kind.
According to the invention, there is provided a license plate reading method comprising:
a dictionary forming step of forming a dictionary in which character images of all characters which are used on a license plate have been registered;
an image pickup step of photographing a running vehicle at a predetermined frame period from a position above a road;
an image memory step of successively storing two images of a present frame image and a previous frame image photographed in the image pickup step;
a vehicle detecting step of detecting a motion of the license plate in a front portion of the running vehicle on the basis of a difference between the present and previous frame images stored in the image memory step; and
a character recognizing step of recognizing all characters of the license plate from an image in a license plate area detected in the vehicle detecting step by collating them with registration character images in the dictionary.
The details of the license plate reading method are substantially the same as those in case of the apparatus construction.